You may need some review in order to complete each week's assignments. In most cases we have tried to duplicate review material from previous courses in this Specialization here. However, there may be some things which we did not account for, so please post comments to the Discussion if you need further assistance beyond the review documents we are providing. A listing of the correct versions for the software we will be using in the Cloudera VM is available at the end of this Reading.

Week 1

Introduction, Background, and Data Exploration

This week you will become familiar with the overarching goals of this Capstone. You will be introduced to a fictitious game called "Catch the Pink Flamingo" and the associated data generated by our development team of data scientists. This is a fascinating data set which provides many different options for exploration. The data set actually consists of 8 separate CSV files containing various types of data designed to simulate many aspects of game play and in-game user activity. You will apply what you have learned in this Specialization on this data set to gain a better understanding of the practical challenges that a big data scientist faces every day.

In Week 1 you will also be doing some data exploration, aggregation, and filtering. We provide some Hands-On readings designed to walk you through some data exploration tasks using Splunk.

Week 2

This week, you will perform classification on the "Catch the Pink Flamingo" game data using a decision tree in KNIME.

Week 3

This week you will learn how to use Spark MLlib to do cluster analysis on the simulated game data.

Week 4

This week we be exploring a somewhat different dataset, simulated chat data, and perform some graph analytics using Neo4j.

Week 5

This week we will be gathering our results together and preparing a presentation and report. No review necessary.

Week 6

This week you will submit your presentation and final report and give yourself a huge pat on the back. Congratulations, you have successfully completed the Big Data Specialization!

Instructions for Downloading Software

Below are the download links for the software used each week.

Week 1: Splunk

https://www.splunk.com/en_us/download/splunk-enterprise.html?ac=coursera_download

Week 2: KNIME

You can skip the survey info in order to download the file directly.

https://www.knime.org/downloads/overview

Week 3: Spark

You can run Spark using the jupyter-coursera Docker container. Instructions for running this container can be found in the following link (steps 1-3):

https://www.coursera.org/teach/big-data-integration-processing/E7n7yvYMEe6GXwr_-7f0TQ/content/item/supplement/iqJiA

Week 4: Neo4J

You can run Neo4J using the neo4j-coursera Docker container. Instructions for running this container can be found in the following link:

https://www.coursera.org/teach/big-data-graph-analytics/y1KZN_3AEe6LHQ57CVhCcw/content/item/supplement/sXXe0

Docker:

https://www.docker.com/products/docker-desktop/